Skip to content

Harness phases 4–5: Effect-native code, enforced invariants, HttpApi contract#20

Merged
koomen merged 15 commits into
mainfrom
improve-harness
Jul 20, 2026
Merged

Harness phases 4–5: Effect-native code, enforced invariants, HttpApi contract#20
koomen merged 15 commits into
mainfrom
improve-harness

Conversation

@koomen

@koomen koomen commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Implements Phases 4 and 5 of notes/improve-harness-plan.md plus the stretch item. bun run ci passes in all 13 workspaces, including the three-backend full-loop e2e lanes and the real-Chromium browser security suite.

Phase 4 — Effect-native everywhere

Hand-rolled codecs, JSON parsing, and validation in shared/cli/server delegated to effect/Encoding, Schema, and the Effect stdlib (net −228 product lines). All async/Promise code extracted into six tiny documented boundary files.

Phase 5 — six invariants, codified and enforced

  • Root AGENTS.md (with CLAUDE.md symlink) states the six invariants and the standing verify-before-commit rule; a check-invariants skill covers the agent-pass residue.
  • scripts/check-boundaries.ts mechanizes the Effect-boundary lint, cli⇄server import boundary, and CLI API-surface check.
  • Adversarial token corpus over all four token kinds (hardened verifySignedValue along the way), session-secret length-floor test, and a conformance test guarding the one sanctioned component-scan duplication.
  • Server-owned route-policy registry: every JSON endpoint declares auth mode, minimum role, mutation, and visibility; the policy test matrix is generated from the same registry and denies every unspecified cell.
  • Storage-adapter conformance suites run unchanged against memory/file, D1/R2 (miniflare), and DynamoDB/S3 (LocalStack) — and caught three real adapter bugs.
  • Cross-host browser security suite in headless Chromium: cookie scoping, handoff flow, cross-origin rejection, private-content isolation.

Stretch — the CLI↔server contract is now structural

The JSON API is defined once as an @effect/platform HttpApi object (ScratchworkApi in shared/src/publish/api.ts). The CLI derives its client from it (HttpApiClient), and the server's route registry, request decoding, and response encoding derive from the same object via a mapped-type policy record — an endpoint, method, path, or schema that drifts between the two sides cannot typecheck.

🤖 Generated with Claude Code

koomen and others added 15 commits July 14, 2026 00:55
…dary

Migrate shared/src to Effect-native per invariant 1 (maximal delegation),
update every cli/server call site, and shrink the async/Promise surface to
six small documented boundary files.

- Delete hand-rolled base64/base64url/hex codecs; call sites use
  effect/Encoding. shared/src/encoding/base64.ts retains only
  decodedBase64ByteLength (size limits without allocating the decode; no
  Effect equivalent), with a conformance test pinning agreement with
  Encoding.decodeBase64. toArrayBuffer and errorMessage stay with documented
  rationale.
- Delete util/json.ts (parseJson -> Schema.parseJson decoding, isRecord ->
  Predicate.isRecord) and util/strings.ts (nonEmpty chains are plain ||).
- publish/bundle.ts is now the single Schema definition of the bundle wire
  format: decodePublishBundle deleted, path uniqueness enforced in the
  schema, api.ts imports the schema instead of redefining it.
- Sweep: auth.json validation (cli/src/auth.ts), JWKS env parsing
  (server config.ts), and API error-body sniffing (cli/src/api.ts) become
  Schema decodes.
- Extract the Web Crypto boundary so auth.ts and login.ts contain no
  async/await/Promise: HMAC + AES-GCM -> server/core/src/auth-crypto.ts,
  SHA-256 digests -> shared/src/crypto/digest.ts (shared by cli PKCE and
  server), Google token-endpoint POST -> google-jwt.ts, and the login
  loopback Bun.serve handler -> cli/src/commands/login-callback-server.ts.
- Signed payloads already carried version + kind through the shared codec
  (landed with Phase 3); verified and checked off.

bun run ci passes in all 13 workspaces; net -228 lines.

Co-Authored-By: Claude Fable 5 <[email protected]>
- generateLoginProof: run randomness inside the Effect (Effect.sync) so
  re-running yields fresh state/verifier instead of reusing values
  captured at construction
- google-jwt: decode the token-endpoint response with a tolerant Schema
  instead of an unvalidated cast; malformed bodies become null
- decodedBase64ByteLength: skip the CR/LF-strip copy when the input has
  no CR/LF (the common multi-megabyte publish case)
- auth.json schema comment: state plainly that unknown fields survive
  reads but are dropped by the next write
- LoginCallbackServer.stop doc: stop(false) waits for in-flight
  requests, which is why settleExchange must be called first

Co-Authored-By: Claude Fable 5 <[email protected]>
AGENTS.md is now the sole normative copy of the invariants registry —
workspace map, the one gate, the standing verify-before-commit rule, and
all six invariants in full. The plan links here instead of retaining a
second editable copy, and the examples//notes/ gate exemption is
documented (Phase 6 item).

Co-Authored-By: Claude Fable 5 <[email protected]>
…ventory checks

scripts/check-boundaries.ts runs first in the root ci gate. It lexes
sources (comments and string prose can't fool it) and enforces:

- Effect-boundary lint: async/await/new Promise/.then(/Promise.* appear
  only in the 14 reviewed boundary files, each with its rationale; stale
  or unused allowlist entries fail so the baseline shrinks but never
  grows silently.
- Import boundary: cli never imports server and vice versa, shared
  imports neither, renderer/src imports none of them.
- CLI route inventory: every route literal or helper call in cli/src
  must match the explicit inventory, whose schemas must be exported by
  shared/src/publish/api.ts. The {error} envelope is now
  ApiErrorBodySchema in the shared contract.

Co-Authored-By: Claude Fable 5 <[email protected]>
shared/test/component-scan-conformance.test.ts runs both
collectComponentNames implementations (renderer plain-JS copy and the
shared original) over an adversarial markdown table — nested backtick
runs, fences, comments, prefix slices — and fails on any disagreement.
The duplication stays permitted only while this proves the copies agree.

Co-Authored-By: Claude Fable 5 <[email protected]>
server/core/test/token-corpus.test.ts drives all four signed token kinds
(session, oauth-state, cli-code, project-access handoff+cookie) through
their production verification paths: every single-bit flip, every
cross-kind pairing, missing/extra/wrong-typed fields, boundary and
non-finite timestamps, segment games, non-canonical base64url, oversized
inputs, expiry boundaries, and an explicit statement of which stateless
kinds stay replayable within their lifetime.

Writing the corpus surfaced five gaps in verifySignedValue, now closed:
extra token segments were silently ignored; excess payload properties
were tolerated by the default Schema decode; 1e999 parsed to Infinity
and would have made a token eternal; nothing rejected future issuance;
and non-canonical base64url encodings of a valid payload were accepted.

Co-Authored-By: Claude Fable 5 <[email protected]>
The 32-byte floor existed in readServerConfig but nothing pinned it.
Covers the 31/32-byte boundary, bytes-not-characters (multibyte), and
both auth modes.

Co-Authored-By: Claude Fable 5 <[email protected]>
Every JSON endpoint is now defined exactly once in api-routes.ts —
method/path, auth mode, minimum project role, mutation flag, response
visibility, handler — and dispatchApiRoute is the only way an API
request reaches a handler: origin rejection first, then the declared
principal resolution, then a read-gated project capability handed to
the handler. app.ts keeps browser auth routes and content serving; the
origin/URL helpers moved to http.ts.

test/api-policy.test.ts derives the expected outcome of every route ×
credential kind (none, garbage, stranger/reader/writer/admin/owner) ×
public/private cell from the same registry, and denies everything the
policy does not explicitly allow. Tightenings that fell out: all JSON
routes reject cross-origin browser calls (previously only some
mutations), sub-read callers get the existence mask on every project
route including delete, and deleting a missing project is 404 instead
of silently ok.

Co-Authored-By: Claude Fable 5 <[email protected]>
…caught

One PrimitiveDb suite and one ObjectStorage suite (invariant 6), run
unchanged against every implementation: memory (server/core), local
file (server/deploy-local — its first real test), D1/R2 under miniflare
and DynamoDB/S3 against LocalStack (e2e). Coverage: JSON/binary round
trips, conditional create/update/delete conflicts, version and ETag
behavior, UTF-8-byte-order listing with exclusive cursor pagination,
expiry semantics, key/namespace validation, error-tag mapping, and
concurrent single-winner races.

First run caught three real divergences, all fixed:
- DynamoDB list without a prefix always threw ValidationException — the
  adapter passed the #key expression name even when no expression used it.
- LocalObjectStorageLive's conditional writes awaited between the
  precondition read and the write, so every concurrent conditional
  create won; a write semaphore makes read-check-write atomic.
- The in-memory test storage had the same TOCTOU, plus unvalidated keys
  on reads.

Co-Authored-By: Claude Fable 5 <[email protected]>
e2e/test/browser-security.test.ts drives headless Chromium (system
Chrome when present, playwright-managed otherwise) against the
local-dev backend on three real origins — localhost app,
pages.localhost content, home.localhost homepage — asserting what only
a real browser can prove: the session cookie is host-only/HttpOnly/Lax
and never crosses to the content host on the wire; handoff cookies stay
path-scoped; published JavaScript cannot plant or override an app
session (parent-domain plants die on the *.localhost public-suffix
rule, forged same-host cookies die on signature verification); other
projects cannot load private content as subresources (real Referer +
Sec-Fetch semantics); cross-origin form POSTs are rejected by origin
policy before auth; returnTo redirects never leave their origins; and
the private homepage origin is isolated from content-host pages.

Co-Authored-By: Claude Fable 5 <[email protected]>
Diff against main, judge the changed files against each invariant's
agent-pass bullets (the calls CI cannot mechanize), report
obeys/violates with file:line evidence, and end with a pass/fail
verdict. AGENTS.md stays the normative copy; the skill only encodes the
procedure.

Co-Authored-By: Claude Fable 5 <[email protected]>
bun 1.3 exits 1 when a workspace has zero test files, where the pinned
1.2.15 exits 0 — so the gate failed on newer local Buns in the four
per-domain deploy projects that legitimately have no tests.
--pass-with-no-tests is accepted by both versions and keeps the
behavior identical.

Co-Authored-By: Claude Fable 5 <[email protected]>
The three-column table was far too wide to read in a terminal; the
package column carried almost no information since the names follow one
rule. A nested list keeps every line readable.

Co-Authored-By: Claude Fable 5 <[email protected]>
The invariant-2 structural fix: the JSON API is now defined once as an
HttpApi object (ScratchworkApi in shared/src/publish/api.ts) that both
sides derive from, so contract drift cannot typecheck.

- shared: ScratchworkApi declares all 11 endpoints (method, typed
  :project path param, payload/urlParams schema, success schema). New
  shared schemas: MeResponseSchema, ShareRequestSchema (moved from
  server), OkResponseSchema, ProjectBundleResponseSchema.
- server: API_POLICY is a mapped-type record keyed exhaustively by
  contract endpoint names; bodies are size-capped and strictly decoded
  through the contract payload schemas in one dispatcher path, and each
  handler's return type is compile-checked against and encoded through
  the endpoint's success schema. The bespoke dispatcher stays:
  HttpApiBuilder's router semantics (400 on bad path params, 404 on
  method mismatch, its own error envelope) conflict with the
  masking/405/origin behavior the policy matrix pins.
- cli: the client is HttpApiClient-derived; bearer/CF-Access headers,
  edge-block detection, and {error}-envelope extraction live in one
  transformed HttpClient, and commands map failures via mapApiErrors
  with unchanged user-facing messages. Malformed project names fail
  fast at resolveProjectRef.
- gate/docs: check-boundaries section 3 becomes the no-hand-built-URLs
  check (allowlist: the /auth/login browser navigation); AGENTS.md
  invariant 2 updated from planned to landed; plan stretch item closed.

Also commits the regenerated default-renderer.generated.js, stale since
the lockfile change (the renderer source hash covers bun.lock; the
rendered HTML is unchanged).

Co-Authored-By: Claude Fable 5 <[email protected]>
@koomen koomen changed the title Phase 4: Effect-native shared/src — delegate codecs, JSON, and validation to Effect; extract the crypto boundary Harness phases 4–5 + HttpApi contract: Effect-native code, six enforced invariants, structural CLI↔server contract Jul 20, 2026
@koomen koomen changed the title Harness phases 4–5 + HttpApi contract: Effect-native code, six enforced invariants, structural CLI↔server contract Harness phases 4–5: Effect-native code, enforced invariants, HttpApi contract Jul 20, 2026
@koomen
koomen merged commit 6dbaa91 into main Jul 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant